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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timery. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later lhan three months after the mailing date of this communication, even if timery filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)D Responsive to communication(s) filed on . 



2a)D This action is FINAL 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) I3 Claim(s) 1-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) I3 Claim(s) 1-3.6.7.9-14.18 and 19 is/are rejected. 

7) S Claim(s) 4.5.8 and 15-17 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or(f). 

a)|3 All b)D Some*c)D None of: 

1 .£3 Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

1 4) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment!, s) 

1 ) |3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) IS] Information Disclosure Statement(s) (PTO-1 449) Paper No(s) 3. 6) □ Other 
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DETAILED ACTION 

1. Claims 1-19 are pending in this application. Claims 1, 9, 10, 14, 18 and 19 
are independent claims. This action is non-final. 

This application claims foreign priority dated 6/1 1/99. 

2. The present title of the invention is "Marker Layout Method, Mixed Reality 
Apparatus, and Mixed Reality Space Image Generation Method". 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in 
the United States. 

4. Claims 1-3, 6-7, 10-11 and 13 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Crane et al. (6,054,991). 

As per claims 1-3, 6-7, 10-1 1 and 13, Crane et al., hereinafter Crane, 
disclose a marker layout method for laying out markers in a real space as 
position indices upon presenting a mixed reality space, comprising the step of: 

laying out the markers to have a positional relationship that allows a given 
player not to observe markers to be used by only another player when a plurality 
of players who observe the mixed reality space within different movable ranges 
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observe the mixed reality space ("The graphical representation obscures the first 
and second objects according to the relative position of the first and second 
objects to a predetermined observation point within the virtual reality 
environment", column 2, line 10-14, where the objects are the marker). 

5. As per claim 2, Crane demonstrated all the elements as applied to the 
rejected claim 1 , supra, and further discloses a visible feature of the markers to 
be used by only the other player is similar to a visible feature of the markers used 
by the given player (Figure 2, since the different players share the same 
processor and memory, it is anticipated the marker used are similar). 

6. As per claim 3, Crane demonstrated all the elements as applied to the 
rejected claim 1, supra, and further discloses the visible feature includes at least 
one of color, texture, shape, and size of the marker ("Each terminal (or leaf) node 
in the tree specified an object (typically an convex polyhedron) composed of 
some number of polygons", column 1 1 , line 29-1 1 ) . 

7. As per claim 6, Crane demonstrated all the elements as applied to the 
rejected claim 1, supra, and further discloses the markers to be laid out include 
markers shared by a plurality of players (Figure 2 84, the game cartridge 
contains the markers to be shared by all participating players). 

8. As per claim 7, Crane demonstrated all the elements as applied to the 
rejected claim 1, supra, and further discloses the markers have a common color 
("each polygon is specified in terms of its color", column 16, line 51-52, where the 
polygon is of the object). 
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9. As per claim 10, Ralston discloses a mixed reality apparatus for making a 
player experience mixed reality by making the player observe a mixed reality 
space image obtained by mixing real and virtual spaces, 

markers serving as position indices being laid out in the real space, 
said apparatus comprising: 

marker detection means for detecting the markers from image data 
obtained by sensing the real space from a substantially viewpoint position of the 
player ("The field of view generally contains real "objects" which are being 
observed in the environment by the operator", column 9, line 29-31 , where the 
object is the marker); and 

mixed reality space image generation means for generating the mixed 
reality space image to be observed by the player, so the player observes virtual 
object images that do not include any images of the markers in surrounding 
regions (marker regions) including markers in the image data (The field of view 
generally contains real "objects" which are being observed in the environment by 
the operator, or may be hidden from sight", column 9, line 29-31). 

10. As per 1 1 , Ralston demonstrated all the elements as applied to the 
rejected claim 10, supra, and further discloses said mixed reality space image 
generation means substitutes or overlays images of the marker regions by 
predetermined virtual object images (" ... and is augmented with images of virtual 
"objects" which are generated by the rendering system and presented on a 
display 415", column 24-34). 
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11. As per claim 1 3, Ralston demonstrated all the elements as applied to the 
rejected claim 11, supra, and further discloses the predetermined virtual object 
images are plane patches with the same or similar texture, size, location and 
posture of the marker regions before the markers are laid out ("Position and 
attribute information relating to selected real objects in a particular environment is 
stored n a database 420 which is accessed by the rendering system to generate 
the corresponding virtual objects", column 9, line 41-45). 



Claim Rejections - 35 USC § 103 



12. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 

all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

13. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Crane et al. (6,054,991). 

As per claim 9, Crane discloses a computer readable storage medium 
which stores a marker layout program for laying out markers in a real space as 
position indices upon presenting a mixed reality space, storing: 

a program code of laying out the markers to have a positional relationship 
that allows a given player not to observe markers to be used by only another 
player when a plurality of players who observe the mixed reality space within 
different movable ranges observe the mixed reality space ("The graphical 
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representation obscures the first and second objects according to the relative 
position of the first and second objects to a predetermined observation point 
within the virtual reality environment", column 2, line 10-14, where the objects are 
the marker). 

Since Crane is using processor and memory (Figure 2 76, 80) for his 
system, it is obvious that he is using the software stored in the hardware to do 
the method as claimed. 

14. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ralston (6,094,625) 

As per claim 12, Ralston demonstrated all the elements as applied to the 
rejected claim 1 , supra. 

Since Ralston suggested overlay the objects with virtual objects, it would 
have been obvious to substitutes or overlays an image of a region including all 
the detected markers by a predetermined virtual object image. 

15. Claims 14 and 18-19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hirota et al. (6,064,749) in view of Crane et al. (6,054,991). 

As per claim 14, Hirota et al., hereinafter Hirota, discloses a mixed reality 
space image generation method for generating a mixed reality space image 
which makes a player experience mixed reality by mixing a real space in which 
markers serving as position indices are laid out, and a virtual space, comprising: 

the marker detection step of detecting the markers from image data 
obtained by sensing the real space from a substantially viewpoint position of the 
player ("Video tracking of landmarks is utilized as the primary method for 
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determining camera position and orientation", column 3, line 16-18, where the 
landmark is the marker and the camera is the player position). 

It is noted that Hirota does not explicitly disclose the step of generating the 
mixed reality space image to be observed by the player, so the player observes 
virtual object images that do not include any images of the markers in 
surrounding regions (marker regions) including the markers in the image data, 
however, this is known in the art as taught by Crane. Crane discloses a method 
of modeling player position where "the graphical representation obscures the first 
and second objects according to the relative position of the first and second 
objects to a predetermined observation point within the virtual reality 
environment", column 2, line 10-14. 

Thus, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to incorporate the teaching of Crane into Hirota in 
order to make the images look 3-dimensional. 

16. As per claim 18, since Hirota's system has processor (Figure 2 76) and 
memory (80), it is obvious to have the storage medium to store the program as 
claimed and, therefore, is similarly rejected as claim 14. 

17. As per claim 19, since Hirota's system has processor (Figure 2 76) and 
memory (80), it is obvious to have the storage medium as claimed and, therefore, 
is similarly rejected as claim 14. 



Conclusion 
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18. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 



19. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Ryan Yang whose telephone number is (703) 
308-6133. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Michael Razavi, can be reached at (703) 305-4713. 

Any response to this action should be mailed to: 



or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 
Crystal Drive, Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application 
or proceeding should be directed to the Technology Center 2600 Customer 
Service Office whose telephone number is (703) 306-0377. /"^ ) 
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Washington, D.C. 20231 



Ryan Yang 
May 15, 2002 
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